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Abstract — The reconstruction of a deterministic data field 
from binary-quantized noisy observations of sensors which are 
randomly deployed over the field domain is studied. The study 
focuses on the extremes of lack of deterministic control in the 
sensor deployment, lack of knowledge of the noise distribution, 
and lack of sensing precision and reliability. Such adverse 
conditions are motivated by possible real-world scenarios where 
a large collection of low-cost, crudely manufactured sensors are 
mass-deployed in an environment where little can be assumed 
about the ambient noise. A simple estimator that reconstructs the 
entire data field from these unreliable, binary-quantized, noisy 
observations is proposed. Technical conditions for the almost 
sure and integrated mean squared error (MSE) convergence 
of the estimate to the data field, as the number of sensors 
tends to infinity, are derived and their implications are dis- 
cussed. For finite-dimensional, bounded-variation, and Sobolev- 
differentiable function classes, specific integrated MSE decay 
rates are derived. For the first and third function classes these 
rates are found to be minimax order optimal with respect to 
infinite precision sensing and known noise distribution. 

Keywords: nonparametric regression; Monte-Carlo sampling; 
dithered scalar quantization; minimax rate of convergence; 
almost sure convergence; oversampled analog-to-digital con- 
version; distributed source coding; sensor networks; scaling 
law; 



I. Introduction 

In a recent paper [1] we considered the problem of re- 
constructing a bounded deterministic multidimensional data 
field / : [0, l] p — > [a,— a], < a < oo, from noisy 
dithered binary-quantized observations collected by n sensors 
randomly deployed over the field domain. The random sen- 
sor deployment model was based on uniform Monte Carlo 
sampling locations where n sensors are independently and 
identically distributed (iid) uniformly over the field domair@ 
[0, l] p . A simple estimator that reconstructs the entire data field 
from these unreliable, binary-quantized, noisy observations 
was proposed in [1] and an upper bound on the integrated MSE 
of the estimator was derived. Using this bound, the integrated 

'This material is based upon work supported by the US National Science 
Foundation (NSF) under award (CAREER) CCF-0546598. Any opinions, 
findings, and conclusions or recommendations expressed in this material are 
those of the authors and do not necessarily reflect the views of the NSF. 
A part of this work was presented at the 2007 International Symposium on 
Information Theory (ISIT). 

2 The field domain [0, l] p is used for clarity and ease of exposition. 
However, the results can be generalized to compact subsets of R p . 



MSE convergence of the estimator to the actual field as the 
number of sensors n — > oo was established. 

In the present paper we expand and complete the devel- 
opment of results in [1]: (i) In Section IIII-BI we expand 
the results of [1] to general deployment distributions. We 
establish a general upper bound to the integrated MSE which 
highlights the interaction of the deployment distribution and 
the orthonormal basis used for non-parametric field estimation 
(Theorem 13. It . (ii) We then derive sufficient conditions on 
the deployment distribution, the orthonormal basis, and the 
dimension of the field estimate which ensure the asymptotic 
(as n — ► oo) integrated MSE consistency of the proposed 
estimator. Implications for desirable deployment distributions 
are also discussed, (iii) In Section UlI-CI we comprehensively 
investigate the asymptotic (as n — > oo) almost sure consis- 
tency of the proposed estimator. The highlight of this section 
is Theorem 13.21 which provides an interesting set of sufficient 
conditions on the deployment distribution, the orthonormal 
basis, and the dimension of the field estimate which ensures 
asymptotic almost sure consistency of the estimation error. The 
implications of Theorem 13.21 are explored in detail through 
Proposition 13.11 and Corollary 13.21 and are of independent 
interest. 

For the finite-dimensional, bounded-variation, and 
Sobolev-differentiable function classes, explicit achievable 
decay rates for the integrated MSEs are provided in 
Section [IV] Specifically, for fields that belong to a finite- 
dimensional function space, the integrated MSE decays a^| 
0(1 /n) (Corollary 14. Q . For fields of bounded-variation, the 
integrated MSE decays as 0(1/ y/n) (Corollary l4~2l l, For 
fields that are s-Sobolev smooth (see UV-Ct . the integrated 

-2a 

MSE decays as 0(n' 2 '+ 1 ) (Corollary I4.3ll. 

One of the highlights of this work is that for multidimen- 
sional fields living in rich function spaces, the minimax rate of 
convergence, of the integrated MSE, even with randomly de- 
ployed sensors, unknown noise statistics, and binary dithered 
scalar quantization (a highly nonlinear operation), can match 
the minimax rate of convergence with infinite-precision real- 
valued samples and known noise statistics. 

The application context of this work is distributed sensing 
and coding for field reconstruction in wireless sensor networks 
as in [1]. The focus is on the extremes of lack of control in 

3 Landau's asymptotic notation: f(n) = 0(g(n)) 
limsup^^ \f{n)/g{n)\ < oo; f(n) = Q(g(n)) g(n) = 0(/(n)); 
/(n) = B(g(n)) & f(n) = 0(g(n)) and g(n) = 0(/(n)). 
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the sensor deployment, arbitrariness and lack of knowledge 
of the noise distribution, and low-precision and unreliability 
in the sensors. These adverse conditions are motivated by 
possible real-world scenarios where a large collection of low- 
cost, crudely manufactured sensors are mass-deployed in an 
environment where little can be assumed about the ambient 
noise. Each sensor measures a noisy sample of the field at 
its location under iid zero-mean, bounded amplitude, additive 
noise. The statistical distribution of the noise is unknown to 
the sensors and the fusion center, and the results in this paper 
hold for arbitrary distributions satisfying these assumptions. 
Each noisy sensor sample is quantized to a binary value by 
comparison with a random threshold (1-bit dithered scalar 
quantization). The binary-quantization models the extreme 
of low-precision quantization. The random thresholds are 
assumed to be iid across the sensors and uniformly distributed 
over the sample dynamic range, modeling the extreme unrelia- 
bility in the quantization across sensors due to manufacturing 
process variations and environmental conditions at different 
sensor locations. Such extreme modeling assumptions are con- 
sidered to demonstrate what is still achievable under adverse 
conditions. 

The communication channel issues are abstracted away by 
assuming that the underlying sensor communication network 
is able to handle the modest payload of transmitting one bit 
(the binary-quantized observation) per sensor to the fusion 
center. The focus of this work is on reconstructing a single 
time snapshot of the field at a fusion center. The recon- 
struction of multiple time snapshots of the field can also be 
accommodated within the framework of this work as in [2] 
but is omitted for clarity. In fact, this can be achieved with 
time-sharing sensors, vanishing per-sensor rate, and vanishing 
sensor location "overheads'0 (see [2]). It is also assumed 
that the fusion center has access to the physical locations of 
the sensors and can correctly associate messages with their 
points of origin. This may be justifiable by possible models 
for the underlying wireless transmission where triangulation 
of sensors is inherently performed. The problem setup is 
illustrated in Figure [T] 

The available literature on distributed field estimation which 
simultaneously treats binary-sensing, random sensor deploy- 
ment, and unknown observation noise distribution is limited. 
The early works in [3], [4] consider the problem of recon- 
structing a signal from binary-quantized samples acquired 
with random thresholds, but do not consider arbitrary additive 
noise with an unknown distribution and only consider fixed 
deterministic sampling locations (deployment). The work in 
[5] is limited to the estimation of a constant field and does 
not explicitly address sampling precision (sensing) constraints. 
A recent work [2] provides pointwise MSE decay rates in 
terms of the local and global modulus of field continuity 
by building upon the techniques in [3], [4], [5]. However, 
[2] does not consider random sensor deployment and re- 
quires local field continuity for pointwise MSE convergence. 
The present work incorporates random sensor deployment, 

4 Network overheads refer to additional bits of information that must be 
attached to each message to identify the point of origin of the message. 



binary-sensing, and unknown noise distribution while studying 
almost sure and integrated MSE convergence of the field 
estimate. The integrated MSE convergence for the bounded- 
variation, Sobolev-differentiable, and finite-dimensional func- 
tion classes are explored in detail. Our results expose the 
effects of field "smoothness", deployment randomness, and 
observation/sensing noise on the integrated MSE scaling be- 
havior. 

For field estimation approaches which are not con- 
strained by finite sensing precision and sensing unreliability, 
such as those involving "uncoded" analog joint sampling- 
transmission, there is a growing body of literature now avail- 
able (e.g., see [6], [7], [8], [9], [10], [11], [12] and references 
therein). Related to the distributed field reconstruction problem 
is the so-called CEO problem studied in the Information 
Theory community in which the distortion is averaged over 
multiple field snapshots overtime (e.g., see [13], [14], [15] and 
references therein). There is also a significant body of work 
on oversampled A-D conversion (e.g., see [16] and references 
therein), which is loosely related to the results of the present 
work concerning finite-dimensional fields. However, these are 
different problem formulations and are not the focus of the 
present work. 

The rest of this paper is organized as follows. The problem 
formulation with detailed modeling assumptions are presented 
in Section [XT] The core technical results are then summarized 
and discussed in Section [HI] The core results are then used to 
derived explicit expressions of the decay rate of the integrated 
MSE for three rich function classes in Section [IV] The proofs 
of all the core technical results are presented in Section M and 
concluding remarks are made in Section [Vl] 

II. Problem Formulation 

Field Model: We model the field as a real-valued, bounded, 
deterministic function / :T> — > [—a, +a] belonging to a non- 
parametric function class^] T, that is, / € T, where T is 
a set of measurable functions mapping T> to [— a, +a]. The 
domain of the field T> is assumed to be a compact subset 
of W l , the e?-dimensional Euclidean space. The objective is 
to reconstruct this function with high fidelity from binary- 
quantized noisy observations collected by a network of non- 
cooperativ^ sensors that are randomly deployed over the 
domain T>. 

Random Sensor Deployment: We assume that the n sen- 
sors are independently and identically randomly deployed over 
the domain T> according to a known distribution px- If Xi € T> 
denotes the location of the i th sensor for i E {1, . . . , n}, then 
Xi ~ iid px captures the lack of control in sensor deployment. 
We assume that the support of px is T> and that px is a non- 
singular distributiorQ. 

Additive Noise: Each sensor takes a sample of the field 
under additive noise. The noisy samples are given by Yj, = 

5 The number of parameters that specify a non-parametric function class is 
not fixed a priori and is possibly infinite. 

6 The sensors do not exchange information or otherwise collaborate at the 
time of or before taking measurements. 

7 A random variable with a non-singular distribution takes values in a subset 
of T> with Lebesgue measure with probability 0. 
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Fig. 1. Problem Setup: The held (in a single snapshot) is sampled by n sensors at their respective locations under additive noise. Each sample is unreliably 
quantized to a binary value by a comparison with a random threshold. These binary values are transmitted to the fusion center which reconstructs the held. 



f(Xi) + Zi, for i e {1, . . . , n}, where the noise variables 
Zi <~ iid pz and are independent of the sensor locations. We 
assume that each Zi is zero-mean and is bounded in amplitude 
by a constant b > 0, that is, the support of pz is contained 
in [—6, +b\. However, besides these assumed conditions, the 
distribution pz is unknown to either the sensors or the fusion 
center, and the results and methods of this paper hold for 
arbitrary noise distributions satisfying these conditions. We 
let Vz denote the set of all noise distributions satisfying these 
assumptions. Note that since both the field and the noise are 
bounded, the noisy samples are bounded: < c := a + b. 
We assume that the value of c is known. The values of a and 
b can remain unknown to the sensors and the fusion center. 

Unreliable, Binary Quantization: We assume that in the 
sensor hardware frontend, the noisy sample is quantized by an 
unreliable, low-precision analog-to-digital converter. Specif- 
ically, we consider one-bit (binary), dithered, scalar quanti- 
zation implemented as a comparison to a random threshold 
that is uniformly distributed over the sample dynamic range 
[— c, +c]. The binary-quantized observations are given by 

B z = sgn(Yi-Ti) for*G{l,...,n} 

J + l Yi>T h J+l f(X i ) + Z i >T i , 



1, K < Ti. 



-1, f(Xi) + Zi<Ti, 



where T$ ~ iid Unif[— c, +c] are the uniform random thresh- 
olds. The thresholds are independent of the sensor locations 
and the noise. The value of Bi is finally the observation that 
sensor i has access to. 

Transmission: We abstract away communication channel 
issues and assume that the underlying communication net- 
work of the sensors is able to handle the modest payload 
of transmitting one bit per sensor to the fusion center. We 
also assume that the fusion center has access to the physical 
locations of the sensors and can correctly associate messages 
with their points of origin. Thus, we assume that through 
this abstracted communications network, the sensor location 
and quantized observation pairs {(JQ, -Bj)}^_j are reliably 
made available to the fusion center. The reconstruction of 
multiple time snapshots of the field with time-sharing sensors, 
vanishing per-sensor rate and sensor location "overheads" can 



also be accommodated within the framework of this work as 
in [2] but is omitted for clarity. 

Reconstruction and Distortion Criterion: Given 
{(Xi, -Bi)}™ =1 , the fusion center constructs the field estimate 
fxi ...,X n ,Bi ...,B n '■ *D —> C For notational convenience, the 
explicit dependence on {(Xi, Bi)}™ =1 will be suppressed and 
the estimator will simply be denoted by f n . The performance 
criterion is the integrated MSE given by 



£>(/,/„) :=E ||/-/ n 



= E 



|/0r)- f n (x)\ 2 dx 



where the expectation is taken with respect to the random 
noise, thresholds, and the sensor locations. The objective is to 
design an estimator /„ that minimizes the integrated MSE D. 
The problem setup is shown in Figure Q] 

Minimax Integrated MSE: For a given field subclass 
•^sub C of interest are the corresponding upper, lower, 
and minimax rates of convergence of the integrated MSE. A 
positive sequence 7„ is an upper rate of convergence if there 
exists a constant C < oo and an estimator /* such that 



lim sup sup sup 

71 >OC p Z GVz /e^sub 



-'DifJ^KC. 



A positive sequence j n is a lower rate of convergence if there 
exists a constant C > such that 



liminfinf sup sup r ) n 1 D(f,f n ) 

™^°° fn PzZPz /e^sub 



> a 



where the inf^ is the infimum over all field estimators. The 
upper rate represents the asymptotic worst-case performance 
achieved by a given estimator. The lower rate represents 
a fundamental limit on the asymptotic performance of any 
estimator. A positive sequence j n that is both a lower rate 
and an upper rate of convergence is called the minimax rate of 
convergence and the corresponding estimator /* that achieves 
the upper rate is called a minimax order optimal estimator. 

Note that showing D(f, /*) = 0(j n ) for all / S J" su b and 
Pz S Vz for a particular estimator /* is equivalent to showing 
that /* achieves 7„ as an upper rate of convergence of the 
integrated MSE. If it can be further shown that D(f,f n ) = 
f2(7n) for a particular / S J-" S ub> a particular pz S Vz, and for 
all estimators /„, then j n is the minimax rate of convergence 
of the integrated MSE. 
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III. Main Results 

In this section, we describe our proposed field estimator and 
analyze its performance. We show that under suitable technical 
conditions, the field estimate is asymptotically integrated MSE 
consistent, that is, as n — > oo, E[||/ — /„|| 2 ] — > 0. We also 
show that under suitable technical conditions, the field estimate 
is asymptotically almost sure consistent, that is, as n — > oo, 
almost surely f n — > / pointwise almost everywhere on T>. 
We also provide an upper bound to the integrated MSE which 
is used in Section [IV] to derive achievable integrated MSE 
decay rates for specific function classes. The proofs of all 
theorems are presented in Section |V] 

Let T denote the set of all bounded, measurable functions 
f:T>-> [-a, +a]. Note that T C L 2 (X>). Let B = {<j)j}f =0 , 



with 6a :T> 



Z, denote an indexed orthonormal (Schauder) 



basis (e.g. Fourier, wavelet, etc.) of L 2 (D). Any / G T can 
be decomposed as 



f = YAf,4>s 



- E a ^ 



(i) 



where a? :— (/, 6j) denotes the coefficients (projections onto 
the basis functions) of the expansion. The m-term approxima- 
tion of / with respect to an orthonormal basis B = {4>j}JLo 
is given by 



fv 



3=0 



(f^j) 6 3 



(2) 



The corresponding m-term approximation error is given by 

OC oo 

e[f,m,B]:=\\f-f m \\ 2 =J2\(f>^\ 2 =J2\ a rf> ^ 

j=m 



which is a non-negative, non-increasing sequence of m that 
converges to zero for all / E T [17, Chapter 9]. 

A. Proposed estimator 

Our proposed estimator first estimates the first m coeffi- 
cients {ajY^r Q of (Q]i with respect to a given orthonormal 
basis B, according to 



(4) 



for j E {0, ... ,m — 1}. A general tunable field estimate is 
given by the m-term approximation, 



fn 



m — 1 

E & 3 6 3' 
3=0 



(5) 



where m is the tunable design parameter which can be chosen 
to depend on n to optimize the rate of decay of the integrated 
MSE for specific function classes. The final field estimate is 
given by specifying m as a function of n, 



fn • 



i(n)-l 

E - 

3=0 



(6) 



The specification of m(n) for specific function classes is 
discussed in Section [IV] The dependence of m on n needs 



to satisfy certain conditions to ensure that the estimate is 
asymptotically consistent. These conditions are described in 
Section lITLBl and Section lITLCl 

B. Integrated MSE upper bounds and convergence results 

The following theorem, whose proof appears in Sec- 
tion [^A] upper bounds the integrated MSE as the sum of two 
terms. The first term is due to the variance of the coefficient 
estimates. The second term is due to the bias caused by the 
finite-term series approximation. 

Theorem 3.1: (Integrated MSE Upper Bound) Let T, V z , 
and px be as given in Section 1771 Let f n . rn be given by (@ and 
(0, where B = {4>j}J^ is any orthonormal Schauder basis 
of L 2 (X>). Then, V./ € T and \/p z € T>z, the integrated MSE 
is upper bounded by 



D 



11/ /n,m|| 



< 



2 m-1 

-E 

n ^ 

3=0 



px{x) 



dx + s[f, m, B], 



(7) 
non- 



where e[f, m, B], given by (O, is a non-negative, 
increasing sequence that converges to as m > 00. 

In light of Theorem 13.11 we now examine conditions on 
m(n), B, and px which ensure that the estimator is asymptot- 
ically consistent in the integrated MSE sense, that is D — > 
as n — ► 00. The following corollary specifies conditions that 
immediately ensures that the integrated MSE converges to 0. 

Corollary 3.1: (Integrated MSE Convergence of the Field 
Estimate) Under the same setup of Theorem 13.71 ;/ m(n), 

00, as n ► cxi, (8) 



13- 
m(n) 



m(n) — 1 

- E 

3=0 



Px(x) 



■dx 



0, 



00, (9) 



then the estimate converges in the integrated MSE sense to the 
field, that is, 

V/ € T and \/pz € Vz, D — > 0, as n — ► 00. 
Condition ^ is sufficient (and often necessary) to ensure 
that e[f,m,B] converges to 0. Condition (O is equivalent to 
the first term of the integrated MSE upper bound, given in 
@, converging to 0. For some deployment distributions px, 
condition (O may not be attainable for many orthonormal 
bases. For example, let the domain T> = [0, 1] with the 
deployment distribution px(x) = 2x over [0, 1]. Then for any 
orthonormal basis in which 6q{x) = 1 over [0, 1], e.g., Fourier, 
Harr wavelets, Legendre polynomials, etc., the first term of the 
summation in (O is given by 



\Mx)\' 



■dx 



— dx 
2x 



iv Px{x) jo 
Thus integrated MSE upper bound becomes useless. This 
implies that in general the deployment distributions and or- 
thonormal bases have to be appropriately matched as a design 
consideration in order to satisfy condition (|9]). However, condi- 
tion (O is ensured for any orthonormal basis if the deployment 
distribution px has a strictly positive infimum over T>, that is, 



inf px(x) = v > 0. 
xev 



(10) 
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Sensor deployment distributions over compact domains which 
are useful for high-resolution field reconstruction would satisfy 
such a condition. Given ( fTOb , we have that 



m — 1 

E 

3=0 



V 



Px[x) 



dx < 



E 

3=0 

771 — 1 1 

E 1 

3=0 



\^{x)f 



■dx 



m 

V 



and V/ £ f and \/pz E Vz, the corresponding integrated 
MSE upper bound becomes 



D = E 



11/ fn,m || 



■ oo when 



The upper bound in ( fTTT i converges to as n 
m(n) satisfies the following two conditions 



m(n) - 
m(n) 



oo, as n 
> 0, as n - 



» oo, 
oo. 



C. Almost sure convergence results 

In this subsection we establish sufficient conditions for the 
field estimate to be asymptotically almost sure consistent, 
that is, as n — ► oo, almost surely /„ — ► / pointwise 
almost everywhere on T>. First, we establish a key theorem 
that gives sufficient conditions for the convergence of the 
pointwise errors of the estimate with respect to the truncated 
approximation of the field. The proof of this theorem appears 
in Section EE 

Theorem 3.2: (Almost Sure Convergence of Estimate Er- 
rors) Let px be the deployment distribution described in 
Section 1771 satisfying ( 1701 ), B = {4>j}'jLo be an orthonormal 
Schauder basis ofTj 2 (T>), / ra , m be the field estimate given by 
(0 and @, and f rn be the m-term approximation to the field 
given by (|2j. Let 

for all x € T>. If there exists a non-negative, increasing 
sequence of real numbers {A TO }^ =1 , and a non-negative, 
increasing sequence of positive integers {m(n)}'^L 1 which 
satisfy the following three conditions 



Vx, y E T>, 
V/ S T,Vx a E V, 



771—1 1 

^ Px(y) 



<CxA m , (12) 



3=0 



Ve > 0, ex P 



A 2 , , 

m(n) t 



< C 2 A ri 



< oo, 



(13) 



(14) 



where C\ , C2 > are some constants, then V/ E T and Wx E 

T> except on a set of Lebesgue measure zero, as n ► 00, 

almost surely, 



0. 



Conditions (TT~2T > and ( fT3l impose constraints on the basis 
functions B = {0j}°^ o and the deployment distribution px- 



Condition ( fT~4-b implies that as 



A 



m(n) 



jn 



0. 



This places a constraint on how fast m(n) can go to infinity. 
In particular it requires that in relation to A m , m(n) not grow 
too fast with n. 

We now examine some special choices of {A TO }^ =1 for 
which conditions ( fT~2l > and dT3l will hold. For m E {1,2,.. .}, 
define auxiliary functions: 



1 I 

9m ( X ,y) ^ — g^ja,)—, 

_^ m— 1 



(15) 



(16) 



3=0 



for x, y E T>. The following proposition, whose proof appears 
in Section IV-CI gives two sets of conditions on {A m }^ =1 , 
B = {<t)j}j^ , and px, for which conditions ( fT2l > and ( fT3l ) 
will hold. 

Proposition 3.1: Let {A m }^ =1 be as in Theorem \3.2\ and 



9n 



h m be given by ( 1751 ) and ( 1761 ) respectively. 



(i) If 



m 
y\2~ 



0, 



as in 



00, 



(17) 



and for x,y ET> almost everywhere, the limits 



9oo(x,y):= lim g m (x,y), (18) 

m >oc 

hoo (x) := lim h m (x) (19) 

m — > 00 

exist, then the limits are zero almost everywhere and the 
conditions ( 1721 ) and ( 1731 ) are satisfied for some constants 
d,C 2 >0. 

(ii) If the basis functions are uniformly amplitude bounded, 
that is, Vj E {0, 1, • • .} and Vx E T>, 

\$j{x)\ < (3 < 00, 

then conditions ( 1721 ) and ( 1751 ) are satisfied for K m = m 
with constants C\ — /3 2 /v and C2 = a/3yvol(P). 

Part (i) of Proposition 13.11 shows that if the limits of the 
auxiliary functions ( fT3T > and ( fT6] i as m — ► 00 exist, then 
for any A m such that ( fTTI i is satisfied, e.g., A m = m 7 / 2 , 
for any 7 > 1, conditions (fT2l and ( fT3l ) are satisfied for 
some constants. Part (ii) of Proposition 13.11 shows that if the 
basis functions are uniformly bounded as, for example, in the 
orthonormal Fourier and Legendre bases, then conditions (fT2l i 
and ( TT3l l are satisfied for A m = m and given constants. 

We now examine conditions on {A TO }^ =1 under which (fT4b 
will be satisfied. According to Ermakoff's series convergence 
test [18], if for some non-negative, non-increasing, real func- 
tion q(t), t>l, 

e*?(e*) 



lim 



?(*) 



< 1, 



where e is the base of the natural logarithm, then 



00 

E 



q(n) < 00. 



6 



Let q(t) = exp (^r 1 ), t > 1, where ip G (0, 1) and e > 0. 
Then 



e'g(e') e t exp(^) 



= exp (t - eV^ - eh 1 ^ 



q(t) exp(^) 
as i — > oo. By Ermakoff's test, for all ip G (0,1) and all 



e>0, 



E ex p 



71=1 



9 



< oo. 



Thus condition ( fT~4-b will be satisfied if A 2 ^^ = ri^ for any 

t^e (0,1). 

Combining the above result with Proposition 13.11 yields 
possible forms of the design parameters {m(n.)}^_ 1 and 
{A m }^ =1 such that the conditions for almost sure conver- 
gence (fT2l >. (fOT l. and (fT~4T > are all simultaneously satisfied. 

mT/ 2 



Choosing m(n) = 0(n*"), where ip G (0, 1), and A J( 



for some 7 G (1,1/0), yields A 



,(n) 



, where ?//' 



71/) G (-0, 1), which satisfies (fl4] i and ( fTTI i simultaneously. 
With these choices, Proposition 13.11 shows that conditions 
([T2l and ( fT3l will be satisfied as well if the limits ([T8| > 
and (TT~9b of the auxiliary functions (fT~5T > and ( TToT l respectively 
can be assumed to exist. Thus, for any m(n) of the form 
m(n) = 0(n^), where ^ G (0, 1), we can choose {A m }^ =1 
such that conditions (V2\ , (fT3T l. and ( TT4l > are simultaneously 
satisfied, if the limits ( fT~8T > and ( fT9b exist. 

Due to the properties of an orthonormal basis, as m — > 00, 
the m-term approximation, f m given by ©, converges in L 2 - 
norm to / for any / G T . Although, it is not guaranteed 
that for general orthonormal bases f m will converge pointwise 
almost everywhere to a specific function. However, if f m does 
converge almost everywhere to some /oo, then /oo must be 
equal to / almost everywhere. This can be seen by writing 







< 



\f(x)-f 00 ( X )\ 2 dx 



lim inf \f{x) 



liminf / \f(x) 



f m (x)\ 2 dx 
f m (x)\ 2 dx = 0, 



where the inequality follows due to Fatou's lemma [19]. Thus 
f v \f(x) - foo(x)\ 2 dx = 0, so \f(x) - fcoix)] = for x G V 
almost everywhere. For example, it is well known that for any 
/ G T C L 2 ([0, 1]) the m-term Fourier series approximation, 
f m converges to / almost everywhere [20]. 

Corollary 3.2: (Almost Sure Convergence of the Field 
Estimate) Within the context of Theorem 13.21 ;/ conditions 

(113, ( liil ), and M4\ hold and if as m > 00, the m— 

term approximation f m converges almost everywhere to some 

a.c. 

function /oo, then \fx G T>, the pointwise error of the field 
estimate satisfies 



\U{X) ~ f(x)\ 



Thus, for x G 
surely f n (x) — 



< 



\fn(x) - f m{n) (x)\ + \f m(n) (x) 

\ S n(x)\ + \fm{ n ){x) - f(x)\ 
a.s. n 

> (J, as n > 00. 



T> almost everywhere, 
- f(x). 



00, almost 



IV. Achievable Integrated MSE Decay Rates 

In this section, we use the integrated MSE upper bound 
© derive explicit expressions for the achievable upper rates 
of convergence of the integrated MSE for three specific 
function classes, namely, finite-dimensional Ts h , bounded- 
variation Fbv, and s-Sobolev differentiable T s . Throughout 
this section, we assume that ( TTOb holds. 

The general approach for deriving such rates of convergence 
for functions living in a function class JF su b Q T is select an 
appropriate basis B = {<fij}J^ in which the m-term approxi- 
mation error given by e[f, m, B] in © can be upper bounded 
by an explicit function of m for all / G ^sub- Then m in (0 
can be chosen to depend on n to optimize the convergence rate. 
Thus given the appropriate function approximation theoretic 
results that upper bound e[f,m,B], this approach establishes 
achievable upper rates of convergence of the integrated MSE 
for the corresponding function class. 

A. Functions in a finite-dimensional subspace of T 

The first function class represents the scenario where the 
fusion center has an exact prior knowledge of the finite- 
dimensional space in which the function lives. Let T& k denote 
the subset of T that is composed of functions that are linear 
combinations of a given set of k orthonormal functions Bk = 
{<t>j})Zl Note that for any / G T Bk , f = EjZo (f , ^) ^ ■ 
Thus the function approximation at the truncation point m = k 
is exact, that is, f m = f for rn = k so that, 

V/6f Bi , e[f,k,B k ]=0. (20) 

Combining (0 with d20l > yields the following corollary. 
Corollary 4.1: (Decay rate of integrated MSE for TbO Let 

Bk and J-Q k be as given above and Vz and px be as given 
in Section\n\ Let f n ,m be given by © and (0 with Bk as the 
basis. If px satisfies ( 1701 ), then V/ G J~B k and Vpz € Vz, the 
integrated MSE of f n ,m with the truncation point m set to k 
is upper bounded as follows 



D = E 



\\f fn,m || 



< — = o(- 

nv \ n 



Therefore, V/ G Tj3 k and ^pz G Vz, an achievable upper 
rate of convergence of the integrated MSE for fields in a finite- 
dimensional subspace is given by 

1 



D = E 



\\f fn,m || 



o 



It should be noted that for this function class, the field 
estimation problem for integrated MSE is equivalent to a 
finite-dimensional parameter estimation problem with condi- 
tionally independent noisy observations. Under the choice of 
an appropriate, well-behaved noise distributior@ pz G Vz, 
the Cramer-Rao lower bound for the integrated MSE decay 
rate for finite-dimensional parameter estimation from iid noisy 
observations asymptotically behaves as D = 0,(1/ n) for 
all asymptotically integrated MSE consistent estimators [21]. 
Hence the estimator is minimax order optimal for Ts k and 
achieves the minimax rate of convergence j n — (1/n). 



8 A noise distribution is chosen such that the observation model satisfies the 
Cramer-Rao regularity conditions [21]. 
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B. Functions of bounded-variation on Domain T> = [0, 1] 

Let Tbv denote the subset of T which is composed of 
functions on T> = [0, 1] of bounded-variation. Formally, 

- 1 \f(x) -f(x- S)\ 



feT 



lim 

<5^0 



-dx < +oo 



A function in Tbv nas a derivative (at points for which 
it exists) which is uniformly bounded and the sum of the 
amplitudes of its discontinuous jumps is finite. The bounded- 
variation condition represents a minimal "smoothness" as- 
sumption since a restriction is placed on the amount of total 
discontinuous jumps. 

It is well known that for the Fourier basis, 



& 



Fourier 



^0) 




j even, 
j odd 



3=0 



(21) 

the m-term approximation error <£3j is upper bounded as 
follows, 



V/ £ T BV , 



e[f, m,£>Fourier] < — , 

m 



(22) 



where a > is a constant [17, Chapter 9]. Combining ((TJ 
with ( 1221 yields the following corollary. 

Corollary 4.2: (Decay rate of integrated MSE for Tbv) 
Let Tbv be as given above and Vz and px be as given in 
Section |77] Let f n . m be given by (@ and Q with Z?Fouricr 
as given in H21\) . If px satisfies \10h then V/ € Tbv an d 
Vpz G Vz, the integrated MSE of f n , m is upper bounded as 
follows 



D 



11/ /n,m| 



< 



where a > is a constant. Setting m(n) = *Jn to optimize the 
decay rate of the upper bound yields the following achievable 
upper rate of convergence of the integrated MSE V/ G Tbv 
and Vpz € Vz-' 

II / fn,m | 



D 



O 



C. Sobolev differentiable functions on Domain T> = [0, 1] 

This function class includes functions which are differen- 
tiable in a generalized sense to a degree of differentiability 
parameterized by s which can take non-integer values. The 
value of s can be considered as a measure of smoothness. For 
s > 1/2, let T s denote the subset of T which is composed of 
functions on T> = [0, 1] that are s-times Sobolev differentiable. 
Formally, 



T s :={feT 



M 2s |/M| 2 ^<+c 



(23) 



where f{uS) denotes the Fourier transform of /. Note that 
the condition in (l23l (for integer values of s) corresponds to 
the s th derivative of / belonging to L 2 ([0, 1]). Thus, this set 
includes functions that are [sj -times differentiable. 
It is well known that for s > 1/2, 

a 



yfeT s , e[f,m,B Fo m ■icrj _ 



-,2s ' 



(24) 



where a > is a constant [17, Chapter 9]. Combining (0 
with (l24l yields the following corollary. 
Corollary 4.3: (Decay rate of integrated MSE for T s ) Let 

T s be as given above andVz and px be as given in SectionUH 
Let f n ,m be given by (01 and (0 with BFourier as given in 
( T271 ). If px satisfies f fTQl ), then V/ € T s and Vpz S Vz, the 
integrated MSE of f n>m is upper bounded as follows 



D = E 



11/ fn,m\\ 



< 



III/ 



m 



2,s ' 



where a > is a constant. Setting m{n) = n 23 + 1 to 
optimize the decay rate of the upper bound yields the following 
achievable upper rate of convergence of the integrated MSE 
V/ € T s and Vpz € Vz'- 



= O ( n 2 =+! 



D=E[||/-/n, m 

It is well known that the exact minimax rate of convergence 
of the integrated MSE for non-parametric regression, based 
on full-resolution, real-valued, noisy observations in an s- 

-2s 

Sobolev space is given by j n = n 2 ^ 1 [22], [23]. In non- 
parametric regression, the field estimate is based directly on 
the full-resolution real-valued noisy observations {li}™ =1 , 
whereas in our problem the field estimate is based on only 
the binary-quantized observations {Bi}f =1 . In both setups, 
the corresponding sensor locations are known. Thus, it is 
interested to observe that our proposed estimator is minimax 
order optimal even with respect to the case in which the 
observations have not been quantized. 

V. Proofs 



A. Proof of Theorem 13.71 

We first establish some results regarding the estimated 
coefficients of @. 

Lemma 5.1: The expected value of an approximated coef- 
ficient is given by 

(i)E[a j ]=a j = (f,<p j ), (25) 
and the integrated MSE of the coefficient estimates satisfies 



(ii) E[\a 3 -a,j\ 2 ] < 



n Jv Px(x) 



dx. 



(26) 



The approximated coefficients also have the following conver- 
gence property 

(Hi) ctj ay, n — > oo. (27) 
Proof: (i) The expectation of the coefficient estimates 
can be evaluated as follows 



E[oy 



Px{X t ) 



sgn(/(J5Q) + Zi - Ti) 



= cE 



sgn(/(Xi) +Zx- T x ; 



, (28) 



where the last equality follows since the terms are iid. This 
last expectation can be evaluated as follows 



E 



sgn(/(AT) + Z x - Ti) 



Px{X x ) 

b r+ c i 4>*Ax) 

Px{x) Pz(z) / - J , , sgn(/(x) +z- t)dtdzdx 

P J-b J-c 2cpx{x) 

+b 1 / rf(x)+z r+c \ 

pz(z)cj)*(x)— / dt — dt\ dzdx 

2c V./_ c / 



P J-b 



1 



+6 



p z (z)</>*(x)(/(x) + z)dzdx 



- / 4>Ux)f(x)dx 



V 



= -(f,<f>j) = 



(29) 



where the second to last line follows from the assumption that 
pz is a zero-mean distribution. Combining (l28l l and (|29l , we 
have (1251 1, (ii) Thus, 



E[|dj - E[dj 



Var[dj 



(30) 



Using standard properties of variance and the fact that the 
terms {<fi*(Xi)Bi/px(Xi)}™ =1 are iid, we obtain the follow- 
ing 



Varfoy 



< 



c 2 


n 


n 2 


E 


i- 


=i 


c 2 








Var 


n 






c 2 








E 




n 






c 2 


E 




n 






c 2 






n 


i 
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cr 






n 


i 






B; 



Bi 



Px(Xi) 



Si 



cr 
n 



E 



<^(*i) 



Bi 



Px(*i) 

l<M*)l 2 

Px( x ) 



p x (x)dx, 
dx. 



(31) 



Combining ( f3Qb and (l3"TT i, we arrive at d26l l. (iii) The coefficient 
estimates 

c^4>*j{Xi) 

a, = — > — -Bi 

= ~ E /y^ ffl 



cE 



sgn(/(Xi) + Z x - Ti) 



(32) 



as n — > oo by Kolmogorov's strong law of large numbers 
since each term in the summation is iid and has a first moment 



bounded by y/vo\(T>): 



E 



Px{X x ) 



= ll^-llx < v^ol(P)||^|| 2 = JvdUpj, 



where the last inequality follows from the Cauchy-Schwartz 
inequality. Combining (1291 and d32l ). we obtain d27l ). conclud- 
ing the proof of the Lemma 15.11 ■ 
For any orthonormal basis B = {</>j}|^ and for any field 
/ G T, the integrated MSE of the estimate can be written as 
follows 

D = E[||/-/ n , ro || 2 ] 



E 



m — 1 



E Q ^ - E 



< 



m—1 

E E ti^ 

3=0 
2 m-1 

-E 

3=0 



p 



+ E m 

2 00 



-dx+^\a 3 \ 2 1 (33) 



where in the last step we used the bound given in 
we have @, concluding the proof of Theorem 13.1 



Thus 



B. Proof of Theorem 13.21 

The pointwise errors of the field estimate with respect to 
the m-term approximation can be written as 



S n {x) 



fn[x) fm(n){x) 

m(n)-l / n /*/ v \ , , 
/ I (A ; )L, 

3=0 

n m(n) — 1 



?i E 



1 



7i 



E E 

i=l j=0 



1 

n ^— f 

i=l 



^ Px{Xi) 

c^*(Xi)Bi 
Px(Xi) 




where for i £ {1, . . . , n} 

Ui(x) : 



3=0 



Px{X, 



Otj <j)j(x). 



Note that Ui(x) is iid and that it is zero-mean due to ([251 ) 
of Lemma IBTTl However, almost sure convergence of S n (x) 
cannot be directly deduced from the standard strong law of 
large numbers since the distribution of Ui(x) itself depends 
on n because it is the summation of m(n) terms. Instead, 
we leverage a more fundamental condition for almost sure 
convergence [24, p. 206]: if for all e > 0, 



X>[|S n (aO| > e] < oo, 



n=l 



then S n (x) --— + as n — > oo. 

Associated with S n (x), is a martingale {Vfc(x)}J?_ given 
by Vq := 0, and for k € {1, . . . , n}, 



k 

V k (x) ^^-U.ix). 



i = l 
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Vq(x), . . . , V n (x) is a martingale since {[/i(x)}" =1 is iid with 
zero-mean. Note that V n (x) — S n (x) and that |Vfc(a;) — 
V k -i(x)\ < \hU k {x)\. For each * G {1, n}, 



\Ui(x)\ < c 



Px{Xi 



m(n) — 1 

E 

3=0 



oij(t>j(x) 



by the triangle inequality. Under the assumptions that the 
conditions given by (fT2] i and ( f]~3T > hold and that the deployment 
distribution px is non-singular, there exists some constant 
C > such that for alii E {1, . . . , n}, 

|Ui(x)| < C*A m(n) , 

with probability 1 for x E T> almost everywhere. Thus 

C-A-m(n) 



\V k (x) - V k -l(x)\ < 



(34) 



According to the Azuma-Hoeffding inequality (see [25, 
p. 303]), if for all k G {1,. .. ,n}, \V k (x) - V k ^{x)\ < C k , 
then for all e > 0, 



P[|K(x)|>e] < 2exp 



Applying this inequality with C k = CA m ( n )/n for all k E 
{!,..., n} (see (l34l ) and V n (x) = S n {x) we obtain the 
following upper bound 



n\S n (x)\ > e] < 2exp 



C 2 A 2 



2 exp 



(. 



^J2k=l „: 



I 2C 2 A 2 M ' 

\ m(n) / 

for xGD almost everywhere. Therefore, 



oc OO / 



m(n) ( 



which is less than infinity Ve > and x E T> almost 
everywhere, due to condition ( fT4V Thus, as n — > oo, almost 
surely S n (x) — ► 0, for x E T> almost everywhere. ■ 



C. Proof of Proposition \3.1\ 

Part (i): Note that if \goo( x ,y)\ = for x,y E V almost 
everywhere and |ft.oo(a;)| = for all / G T and for x E T> 
almost everywhere, then conditions (fT~2T > and ( fT3l hold with 



some constants C\, C% > 0. For g m , we can write 

Px(y)\9oo(x,y)\ 2 dxdy = 



VxV 

lim inf p\ (y) \g m (x, y)\ 2 dxdy 
IVxV m — ,oa 

(a) ff 2 

< liminf// p x (y)|.g m (a;,y)| dsdy 

m »oo 



m— 1 m — l 



i 



j=0 k =0 



4>l{x)(t>k{y) 



p 2 x(y) 



dxdy 



m — l m—l 



^ I I I ± lib J. r, 

M^afE E JAxWA 

m „-_n 7, n JV 



x)dx • 



3=0 fc=0 



<t>k(vWAy)dy 



lim inf —5- 1 = lim inf . , 



m 



where the inequality (a) is due to Fatou's lemma [19] and S k 
is the Kronecker delta function. Thus for A m such that ( TTTb 
is satisfied we have that 



Px(y)\9oo(x,y)\ 2 dxdy = 0, 



VxV 



which implies that \g 00 (x,y)\ = for x,y E T> almost 
everywhere due to ( TTOb . For h m , we can write 



\h nc ,(x)\ dx 



< lim inf 

m — >oo 



lim inf |/i m (a;)| 2 <&E 



m >oo 



|/i m (a;)| 2 (ix 



m — 1 m—l 



= J^inf / tj- E E "i^i^fe^)^ 
m ^°° Jl3 m j=0 fe=0 

^ m — l m—l „ 

= liminf T2~E E a i a *k / <l>i{ x )<t>l( x ) dx 

m^oo A 2 n ^ ^ 



j=0 k=0 



^ m — l 

liminf —5- N 

3 

/II 2 



m »oo A 



< liminf ( 

m — >oo A£ 



< lim inf 



m >oc 



a 2 vol(X>) 
A 1 : 



V/ G 



where the first inequality follows from Fatou's lemma [19] and 
the last inequality is due to / being amplitude-bounded by a 
over the support T>. Thus for A m such that ( TTTb is satisfied, 
we have that 

V/GJ 7 , / ^{x^dx^O, 
Jv 
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which implies that |/ioo(aO| = for all / G T and for x e T> 
almost everywhere. 

Part (ii): Applying the triangle inequality, we can write 

m — 1 _^ 

V (t>j(x)(t>*(y) — -- 

- f^ px{y) 

< y — = — rn = CiA m , 

j=o 

which shows that condition ( TT2t is satisfied for A m = to and 
Ci = P 2 /v. Again, applying the triangle inequality, we can 
write 

m— 1 
m—1 

<£K/,&>ll0i(*)l 

rn— 1 

< E 11/11/3 = HI/11/? 

< maPy/vo\{V) = C 2 A m , 

which shows that condition (IT3l is satisfied for A m = m and 
C 2 = a(iy/vo\{V). ■ 

VI. Concluding Remarks 

The principal contribution of this work is a systematic 
treatment of (i) binary-sensing, (ii) random sensor deployment, 
and (iii) unknown observation noise distribution for high- 
resolution distributed sensing and estimation of multidimen- 
sional fields using dense sensor networks. A key finding of this 
work is that the rate of convergence of the integrated MSE for 
field estimation is extremely robust to the apparent limitations 
of ultra-poor sensing precision, random sensor deployment, 
and lack of knowledge of observation noise statistics. In some 
cases, the convergence rate exactly matches the minimax rate 
of convergence with infinite-precision real-valued samples 
and known noise statistics. Interesting directions for future 
work include (i) establishing the exact rate of convergence 
of the integrated MSE and a central limit theorem for the 
estimate, (ii) analysis of the sensitivity of the integrated- 
MSE to sensor location uncertainty, (iii) unbounded-amplitude 
signal and noise models, and (iv) general dither distributions. 
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